Repository navigation
Measure the width of every team BEAM actually creates - #15
Merged
Merged
Conversation
`test_the_instrumented_build_enables_avx` found the tsan recipe with `^tsan:.*?(?=^\w|\Z)` -- everything up to the next line starting with a word character. Make recipes are tab-indented and the comments between targets start with `#`, so that pattern does not stop at the end of the recipe: it runs through any following comment block and through a following rule whose target begins with `$(`, stopping only at the next target named by a bare word. Found while adding a target between `tsan` and `clean`, which grew the match from 10 lines to 26. The test still passed, but its claim had quietly become "-mavx appears somewhere in this stretch of the Makefile" rather than "the tsan recipe passes -mavx", and it would have gone on passing with the flag moved into a neighbouring recipe and out of this one. Now it matches the target line plus the tab-indented lines under it, which is exactly one recipe. The continuation-collapsing step above it runs first, so a backslash-continued line is already one line by the time this applies. Verified by moving `-mavx` out of the tsan recipe and into the neighbouring one: the test fails, where before the change it passed.
Every thread check in this repository was static. `thread counts` proves all 93 parallel regions NAME a thread count; `thread plan wiring` proves all 69 GIL-releasing regions RESOLVE a plan. Neither can see what the runtime granted, and that gap is not hypothetical: the nesting defect was live in a gate run where `thread counts` reported OK, because the clause was present and correct and only its value on a worker thread was wrong. verification/teamwidth.c interposes the two libgomp entry points BEAM creates teams through -- GOMP_parallel and the fused GOMP_parallel_loop_nonmonotonic_dynamic, the only two of its ten GOMP imports that create a team -- and records, per region entry, the count requested and the width that actually ran, split by whether the region itself is nested. It observes the SHIPPED extension through LD_PRELOAD, with no recompilation. A check that needed its own build would be checking a different binary, which is the same reason the gate hashes the artifact rather than trusting a clean tree. One entropy() call creates 949 teams. That number is the argument for measuring at all: reading 93 pragmas cannot tell you what happens 949 times, and a region inside a loop is one pragma and ten thousand entries. BOTH channels are recorded, and the mutation is why. With the nested-region guard removed from beam_threads(): nesting ON (OMP_MAX_ACTIVE_LEVELS=2) requested 16, nested ran 16, plan 8 nesting OFF (the default) requested 16, nested ran 1, plan 8 So in the environment every user actually gets, the defect was INVISIBLE in the width that ran -- libgomp declines to widen a nested region when nesting is off, so the teams were a blameless 8 and 1 -- while BEAM was still emitting num_threads(16) on them. The runtime's default policy was the only thing between the plan and sixteen-way oversubscription, which is luck rather than design. A check on the width that ran PASSES on that mutant; test_no_region_asks_for_more_than_the_plan fails it with no special environment at all, and is therefore the primary assertion rather than a second opinion. Two tests exist to stop the file lying. One reads the built extension's imports and fails if BEAM grows a third way to create a team, because the observer would then be blind AND quiet -- the worst failure a checking tool has. The other requires the measurement to have reached depth >= 2, since a dead path and a well-behaved one report the same zero; this is the same lesson as the availability estimate, where a quiet runner and a dead reader were indistinguishable. A third reports what the source scan cannot: zero of the 949 regions reached libgomp with nthreads == 0, which is what a missing num_threads clause looks like from the outside. `thread counts` covers regions whether or not they run; this covers regions that ran whether or not the scan understood them. Attribution rests on where the reset goes. LD_PRELOAD interposes libgomp process-wide, so a counter read at exit would mix in any other OpenMP user -- a BLAS built against libgomp, say. The child resets immediately before the one call under test and reads immediately after, with the model built before the reset, so every region counted is BEAM's. Runs in CI as a step in the Verification tooling job, which is the one job that can: it interposes libgomp through LD_PRELOAD, so it is Linux-only by nature. Without `make teamwidth` the tests skip, and a skipped check is not a check. Verified: C 454 tests passing, Python 1243 passed and 2 skipped, tooling 136 passed and 0 skipped, gate exit 0 with every recorded number byte-identical and no inventory line moved -- these tests live in verification/, which the baseline does not track. Both artifacts hash unchanged, the observer being a separate object that the shipped build never links.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last gap in the parallelism work: every thread check in this
repository was static, and none could see what the runtime granted.
thread countsproves all 93 parallel regions NAME a thread count.thread plan wiringproves all 69 GIL-releasing regions RESOLVE a plan.Neither is evidence about a team that ran -- and that is not hypothetical. The
nesting defect was live in a gate run where
thread countsreported OK,because the clause was present and correct and only its value on a worker
thread was wrong.
What it does
verification/teamwidth.cinterposes the two libgomp entry points BEAM createsteams through (
GOMP_paralleland the fusedGOMP_parallel_loop_nonmonotonic_dynamic-- the only two of its ten GOMPimports that create a team) and records, per region entry, the count requested
and the width that actually ran, split by whether the region itself is nested.
It observes the shipped extension through
LD_PRELOAD, with norecompilation. A check that needed its own build would be checking a different
binary, which is the same reason the gate hashes the artifact rather than
trusting a clean tree.
One
entropy()call creates 949 teams. That number is the argument formeasuring at all: reading 93 pragmas cannot tell you what happens 949 times.
What it found immediately
The defect fixed in #14 was latent in every default install, not only under
OMP_MAX_ACTIVE_LEVELS=2as that PR described it. With the guard removed:OMP_MAX_ACTIVE_LEVELS=2)In the environment every user actually gets, the defect was invisible in the
width that ran -- libgomp declines to widen a nested region when nesting is
off, so the teams were a blameless 8 and 1 -- while BEAM was still emitting
num_threads(16)on them. The runtime's default policy was the only thingbetween the plan and sixteen-way oversubscription. That is luck, not design.
Why both channels are recorded
Because the mutation says they are not interchangeable:
test_no_region_asks_for_more_than_the_plantest_no_team_runs_wider_than_the_planAn observer recording only the width that ran is blind to this defect in the
default environment. So the requested channel is the primary assertion and the
actual-width tests corroborate it, rather than the other way round.
The tests that stop the file lying
test_the_observer_wraps_every_team_creating_entry_pointreads the builtextension's imports and fails if BEAM grows a third way to create a team,
because the observer would then be blind AND quiet -- the worst failure a
checking tool has.
test_the_measurement_reached_a_nested_regionrequires depth >= 2, since adead path and a well-behaved one report the same zero. Same lesson as the
availability estimate in Say whether the availability estimate measured the machine #13, where a quiet runner and a dead reader were
indistinguishable.
test_every_region_that_ran_named_a_thread_countreports what the source scancannot: zero of the 949 regions reached libgomp with
nthreads == 0, which iswhat a missing clause looks like from outside.
thread countscovers regionswhether or not they run; this covers regions that ran whether or not the scan
understood them.
Attribution rests on where the reset goes.
LD_PRELOADinterposes libgompprocess-wide, so a counter read at exit would mix in any other OpenMP user -- a
BLAS built against libgomp, say. The child resets immediately before the one
call under test and reads immediately after, with the model built before the
reset, so every region counted is BEAM's.
Second commit
test_the_instrumented_build_enables_avxfound the tsan recipe with^tsan:.*?(?=^\w|\Z), which does not stop at the end of a recipe: adding atarget between
tsanandcleangrew the match from 10 lines to 26. The teststill passed, but its claim had become "-mavx appears somewhere in this stretch
of the Makefile". Now it matches exactly one recipe, verified by moving
-mavxinto the neighbouring target and watching the test fail.
Verification
libbeam.aand the extension hash unchangedNo baseline movement: these tests live in
verification/, which the baselinedoes not track. The observer is a separate object the shipped build never links.
Runs in CI as a step in the
Verification toolingjob -- the one job that can,since
LD_PRELOADover libgomp makes it Linux-only by nature. Withoutmake teamwidththe tests skip, and a skipped check is not a check.